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ABSTRACT 



INTRODUCTION 

This report presents the findings of a formative evaluation of Crossroads Cafe, An ESOL Adult 
Learning Program, which is a distance learning adult level video program designed to teach English 
to Speakers of Other Languages (ESOL). The research focused on how 22 programs were 
implemented in 6 different regions of New York State. Researchers sought answers to what worked, 
what didn't, and why. 

Crossroads Cafe is the product of a collaborative effort of the United States Department of Education; 
the State Education Departments of California, Florida, Illinois, and New York; INTELECOM, a 
501(c)(3) not-for-profit educational television production consortium; and Heinle and Heinle Publishers. 
This formative evaluation of the Crossroads Cafe ESOL Adult Learning Program was conducted by 
the Center for Advanced Study in Education (CASE) of the City University of New York Graduate 
School for the New York State Education Department. Twenty two pilot programs with 755 adult 
learners were part of this study, which was conducted between June 1 and September 30, 1996. 
Evaluation data, in full or in part, were collected from all 22 programs. 

BACKGROUND AND CONTEXT OF THE CROSSROADS CAFE PROGRAM 

The United States is a nation of immigrants, admitting as many legal immigrants and refugees annually 
as the rest of the world combined. The foreign bom population, with 21.2 million people, is larger than 
at any other time in the nation's history and is expected to grow. Limited English Proficient ( LEP) 
adults comprise nearly half of the immigrant population, and by the year 2000 an estimated 
17.4 million LEP adults will reside in the United States. In the last decade, only 633,000 out 
of approximately 4.25 million LEP adults were served by ESOL programs. 

DESCRIPTION OF THE PROGRAM 

Crossroads Cafe is designed to teach English to adult learners working independently, without an 
instructor present. It is targeted to individuals who are literate in their native language and who have 
some proficiency in both spoken and written English. The main component of the learning program 
is a series of 26 half-hour episodes about six ethnically diverse characters whose lives intersect at 
Crossroads Cafe, a neighborhood restaurant. The videos are supported by collateral work units with 
exercises designed to develop story comprehension, language skills, and higher order thinking. Two 
resource books are also available to teachers. A Partner Guide offers suggestions and reproducible 
masters for an English proficient non-professional friend or family member who can guide the learner 
in his/her study of English. 

DESCRIPTION OF THE IMPLEMENTATION EVALUATION STUDY 

Provider agencies with established reputations for prior high quality performance were chosen as pilot 
sites. Because the pilot period was limited to 8 weeks, quantitative pre/post data from pilot sites were 
not assessed in this study. Therefore, experienced, successful, highly respected ESOL administrators 
and instructors were selected to provide expert qualitative assessments of the materials and methods 
employed use din the pilot programs. Pilot sites were also selected on the basis of geographic 
location. Agency participation was voluntary. 

TYPES OF MODELS 

Pilot sites conducted programs that were either [1] a traditional classroom model in which teachers 
used the videos and texts in class with the students, [2] a hybrid model in which the students used the 
materials with the teacher, but the major portion of the viewing and textbook exercises were completed 
without an instructor present, or [3] a distance learning model in which, after an orientation session, 



the students worked essentially alone. Four classroom programs, seven hybrid programs, and eleven 
distance learning programs were conducted at the 19 sites during the pilot period. Distance learning, 
accounting for 50 percent of the programs, was the most prevalent model. The hybrid model was used 
in 32 percent of the programs, and the classroom model in 18 percent of the programs. 

IMPLEMENTATION MEASURES 

The study design incorporated both quantitative and qualitative approaches to data collection and 
analysis, including self-administered questionnaires, personal interviews, and focus groups. 
Information was collected from administrators, teachers, and students periodically during the pilot 
period and/or retrospectively after its conclusion. 

RECRUITMENT AND SELECTION OF ESOL LEARNERS 

Students were recruited for this pilot study primarily from adult education class rolls of the 1995-96 
school year. These classes included ESOL (79%), GED (14%), and ABE (7%). The study population 
mainly included learners who were at the "intermediate" level. Eighty-two percent of all students and 
87% of distance learners remained for the duration of the program. The typical learner was female 
(69%), 37 years old (mean age), Spanish speaking (46%), at least a graduate of a foreign high school 
(71%), a relative newcomer to the United States (median time, 3 years), and unemployed (67%). 

REACTION TO PROGRAM MATERIALS 

The overall reaction to the Crossroads Cafe program was overwhelmingly positive. Teachers rated 
the extent to which learners liked the program materials. Ninety-six percent of the teachers reported 
that students either "liked the materials" or "liked the materials somewhat." Two-thirds of the students 
found the videos and texts "average [in difficulty]” or "not difficult to use.” The Crossroads Cafe video 
series was described by students as "interesting," "enjoyable," "engaging," "compelling," and "very 
positive." Students especially liked the fact that the stories portrayed "real-life" experiences and that 
they could "identify" with the characters and their struggles. 

PROGRAM OUTCOMES 

The Crossroads Cafe pilot study demonstrated the relative ease of establishing ESOL instruction 
programs using these materials in all three program models. Teachers and administrators reported 
that students worked longer hours using Crossroads Cafe materials in the distance learning program 
model than was found among students in other program models. The learners in all program models 
enjoyed their work and eagerly awaited the next episode. In the opinion of the teachers and admini- 
strators, this attitude increased “time on task” in all models and translated into improved acquisition 
of English language skills. In the opinion of teachers, 86.9% of students improved their English 
language skills and increased the accuracy of their workbook activities. Teachers also noted that 
learners acknowledged their own improvement in English language skills and recognized their 
increased accuracy in workbook activities. 

The study clearly demonstrated the feasibility of using Crossroads Cafe as the basis of ESOL instruc- 
tion. Moreover, the use of Crossroads Cafe to promote distance learning as a means of reaching out 
to heretofore unreachable learners was also demonstrated. 

Three major issues need to be resolved before distance or hybrid learning programs can be 
institutionalized: [1] Teachers need to be trained in distance learning methods; [2] Program logistics 
require special attention to facilitate transfer of videos and text materials between instructors and 
teachers; and [3] A state reimbursement formula for service providers must be established that 
adequately and fairly funds school districts, BOCES, and CBO's which choose to conduct distance 
learning programs. 



EXECUTIVE SUMMARY 



I. INTRODUCTION 

This report presents the findings of a formative evaluation of Crossroads Cafe, An ESOL 
Program for Adult Learners, a distance learning adult level video program designed to 
teach English to Speakers of Other Languages (ESOL). The study design focused on 
process: how the program was utilized by administrators, teachers, and learners. The 
research focused on whether the program was successfully implemented, what interfered 
with successful implementation, and why — or simply "what worked, what didn't, and why." 

The Crossroads Cafe program is the result of a collaborative effort of the United States 
Department of Education; the State Education Departments of California, Florida, Illinois, 
and New York; INTELECOM, a 501(c)(3) not-for-profit educational television production 
consortium; and Heinle and Heinle Publishers. 

The evaluation of Crossroads Cafe was conducted by the Center for Advanced Study in 
Education (CASE) of the City University of New York Graduate School for the Office of 
Workforce Preparation and Continuing Education of the New York State Education Depart- 
ment. Twenty two sites with 755 learners participated in this pilot study, conducted 
between June 1 and September 30, 1996. Evaluation data, in full or in part, were collected 
from all 22 sites. 

II. BACKGROUND AND CONTEXT OF THE CROSSROADS CAFE PROGRAM 
A. Historical Background 

The United States is a nation of immigrants, admitting as many legal immigrants and 
refugees annually as the rest of the world combined. The foreign bom population, with 21.2 
million people is larger than at any other time in the nation's history and is expected to 
grow. Limited English Proficient (LEP) adults comprise nearly half of the immigrant 
population and by the year 2000, an estimated 17.4 million LEP adults will reside in the 
United States. As a result, ESOL education has become the fastest growing sector of 
adult education in the country. However, existing programs are unable to meet the ever 
increasing demand for education in this area. 

The foreign-bom population in New York State is among the largest in the country, with the 
majority living in New York City. In the last decade, only 633,000 out of approximately 4.25 
million LEP adults have been served by ESOL programs. 



B. Description of the Program 

The Crossroads Cafe program is designed to teach English to adult learners working inde- 
pendently, without an instructor present. It is targeted to individuals who are literate in their 
own native language and who have some proficiency in both spoken and written English. 



The program materials, comprised of five major components, feature multi-level skills, 
context-based learning, self-instructional methods, communication skills; and multiple 
learning styles. 

The main component is a series of 26 half-hour episodes about six ethnically diverse 
characters whose lives intersect at Crossroads Cafe, a neighborhood restaurant. Each 
video contains a “Wordplay” segment, which demonstrates a specific language function 
and the language structures necessary to do that function, and “Culture Clips,” a docu- 
mentary style segment that examines social or cultural themes contained in the episode. 
In addition, Worktext, the more advanced supplemental text, contains twenty-six learner 
collateral work units with exercises designed to develop story comprehension, language 
skills, and higher order thinking using three graduated levels of exercises. Photo Stories, 
the supplemental text for those with less English proficiency, is a condensed pictorial tour 
of each episode, presented with simple dialogue balloons and four color photographs. 

The distance learning configuration was structured with the anticipation that most learners 
would complete one episode per week. In practice, learners essentially worked at that 
pace, but exceptions were common among those whose English proficiency was relatively 
high. In Ithaca, some highly motivated wives of visiting Cornell faculty, whose English 
proficiency was low, began with Photo Stories and then rapidly moved successfully to the 
Worktext at the rate of three episodes per week. In classroom models, the more structured 
environment appears to have imposed a conformity of pace among learners that facilitated 
class discussion and group process at the expense of individual freedom in determining 
learners’ own rates of progress. 

Two resource books are also available to teachers, which include general instructions on 
program methods, episode-specific suggestions for optional enrichment activities, and 52 
reproducible masters. Finally, the Partner Guide offers suggestions and reproducible 
masters for an English proficient non-professional friend or family member who can guide 
the learner in his study of English. 

III. DESCRIPTION OF THE IMPLEMENTATION EVALUATION STUDY 

A. Selection of study sites 

The Crossroads Cafe program was piloted in 22 programs at 19 demonstration sites in six 
regions of the State. [See Table 1.] Prior experience in providing ESOL instruction to 
adults as well as geographic location were criteria for selection. Provider agencies with 
successful ESOL experience were nominated by NYSED as potential pilot sites. Admini- 
strators at participating sites nominated experienced, qualified instructors. 

B. Implementation models 

Pilot sites selected one or more of three program models for implementation of the 
Crossroads Cafe Program. In the traditional classroom model, teachers used the videos 
and texts in class with the students. In the hybrid model, the students used the materials 
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with the teacher, but the major portion of the viewing and textbook exercises were 
completed without an instructor present. In the distance learning model, after an orienta- 
tion session the students worked essentially alone. Teachers maintained contact with their 
students either by phone or met with them for a very brief exchange period at regular or 
irregular intervals. 

C. Implementation Measures 

The study design incorporated both quantitative and qualitative approaches to data col- 
lection and analysis, including self-administered questionnaires, personal interviews, and 
focus groups. Information was collected from administrators, teachers, and students 
periodically during the pilot period and/or retrospectively after its conclusion. 

IV. Pilot programs 

A. Description of program models 

The type of program model(s) selected for the implementation of the Crossroads Cafe 
Program at each pilot site was based on several criteria. Practical considerations such as 
the availability of funding and geographic distribution of the student population, as well as 
ideological considerations such as a commitment to particular teaching modalities or 
educational philosophies, helped service providers determine their individual program 
designs. Altogether, four classroom programs, seven hybrid programs, and eleven 
distance learning programs were conducted at the 19 sites during the pilot period. 

Distance learning, accounting for 50 percent of the programs, was the most prevalent 
model. The hybrid model was used in 32 percent of the programs, and the classroom 
model in 18 percent of the programs. [See Graphic 2 and Graphic 5.] 

B. Recruitment and selection of ESOL learners 

Pilot program administrators used their own discretion in selecting learners for the pilot 
programs. Because of the short lead time provided by the State Education Department, 
program administrators contacted those for whom they already had names, addresses, and 
phone numbers. As a result, students were recruited for this pilot study primarily from adult 
education class rolls of the 1995-96 school year. These classes included ESOL (79%), 
GED (14%), and ABE (7%). [See Graphic 3.] Students were selected for the various pro- 
gram models based on ESOL proficiency as well as more subjective personal and situa- 
tional factors. All sites provided conveniently scheduled orientation sessions for prospec- 
tive students. 

The study population mainly included learners who were the "intermediate" level with 
regard to ESOL (61%) as measured by BEST or an equivalent test. [See Graphic 4.] 

Overall, the retention rate was high, particularly in distance learning programs. Eighty-two 
percent of all students and 87 percent of distance learners remained for the duration of the 
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program, a remarkable accomplishment given the extent of familial and personal responsi- 
bilities faced by participants. 

C. Profile of ESOL Learners 

The typical learner was female (69%), 37 years old (mean age), Spanish speaking (46%), 
at least a graduate of a foreign high school (71%), a relative newcomer to the United 
States (median time, 3 years), and unemployed (67%). [See Graphics 6 through 12.] 

D. The Implementation of the Crossroads Cafe Program 

The implementation of Crossroads Cafe was relatively easy to initiate, according to pro- 
gram administrators. Using a 5 point scale, where 1 indicated "difficult" and 5 indicated 
"easy," the mean rating for all respondents was 3.3. Administrators in the three models 
responded similarly. [See Graphic 13.] 

The only problem associated with the implementation process was a "late start-up" in about 
half the sites due to insufficient lead time, delayed recruitment of students, and/or limited 
availability of program materials. To expedite the situation involving insufficient videos, 
instructional staff assumed responsibility for duplicating videos for distribution to their 
students, a time-consuming and often costly solution. 

The relative ease of implementation may be attributed to four factors: (1) the caliber of the 
instructional staff, 76 percent of whom were "very experienced" in ESOL instruction [See 
Graphic 14]; the resourcefulness and commitment of the pilot program administrators; the 
perceived quality of the orientation and training conducted by NYSED for administrative 
and instructional staff; and the quality of the Crossroads Cafe video and print materials. 

IV. PROGRAM MATERIALS 

The overall reaction to the Crossroads Cafe program was overwhelmingly positive. Unlike 
more traditional approaches to ESOL, Crossroads promoted "a combination of reading, 
writing, and listening skills training." Teachers rated the extent to which learners liked the 
program materials, and the results were striking. The average rating on a 5 point scale, 
where 1 indicated "dislike" and 5 indicated "like" was 4.5, with 96 percent of all teachers 
saying students either "liked the materials" or "liked the materials somewhat." [See 
Graphic 15.] Mean ratings were similarly high in all program models. 

Clearly the heart of the Crossroads Cafe program was the video series, which was 
described by students as "interesting," "enjoyable," "engaging," "compelling," and generally 
"very positive." Students especially liked the fact that the stories portrayed "real-life" 
experiences and that they could "identify" with the characters and their struggles. 

The use of accents by the main characters was the only feature that had some negative 
implications for students, although most were able to adjust to this in time. Teachers 
generally viewed the accents in a more positive light, as an accurate reflection of our 
society. 



The multi-level Worktext and Photo Stories were viewed by both learners and teachers as 
useful and welcome adjuncts to the video series. 

Not all professional resource materials were available to teachers during the pilot program. 
However, they were able to devise and/or adapt Crossroads Cafe materials and other 
ESOL materials to meet their particular instructional needs. 

B. Utilization of Program Materials 

The Crossroads Cafe materials were designed to be used in a variety of ways — in a 
traditional classroom setting, outside the classroom, and/or in a combination of the two. 
In all programs, a number of factors including personal preference, time and scheduling 
issues, availability of books, videos, and VCR equipment, and program model influenced 
where students worked with the materials. 

Overall, during the six to eight week pilot period, most work, 58% of viewing and 68% of 
reading, occurred outside the classroom setting. [See Graphic s 17 and 18.] As expected, 
nearly all students in classroom programs watched videos in class (89%), while those in 
distance learning programs watched at home (79%). [See Graphic 16.] Just over half of 
classroom students did their reading in class (55%), while most of those in distance 
learning programs did their reading at home (86%). [See Graphics 16 and 19.] 

In spite of the time constraints imposed by the pilot period and the personal constraints 
faced by many learners, a considerable amount of the assigned work was completed. 
When all units are considered, 86% of videos were watched in their entirety and 56% of 
workbook answers were completed in their entirety. [See Graphic 20.] Those in distance 
learning settings completed more units than those in other settings, with 95% watching 
videos completely and 62% answering all questions. 

One indicator of the degree to which learners applied themselves to acquiring English 
language skills is relative "time on task." Overall, the median time per unit was an hour an 
a half. Students in both classroom and distance learning programs spent two hours per 
unit and those in hybrid settings, an hour an a half. [See Graphic 21.] 

Generally, students found the Crossroads Cafe materials relatively easy to use. Based on 
a 5 point scale where 1 indicates "very difficult" and 5 indicates "not difficult at all," the 
average rating for all units combined was 3.2, with two thirds of the units described as 
either "not difficult" or "about average" in difficulty. Students in the three program models 
rated the materials similarly, although they were perceived as slightly easier to use in 
classroom settings. [See Graphic 22.] 

VI. PROGRAM OUTCOMES 

Program staff reported that students improved their English language skills during the 
course of the program. Using a 5 point scale, where 1 indicates "not effective" and 5 
indicated "very effective in helping students improve their skills, administrators rated the 
videos 3.7 and the workbooks, 3.6. [See Graphics 23 and 24.] Using the same scale, 



teachers rated the videos and books together 3.9. [See Graphics 25 and 26.] Teachers also 
noted that learners acknowledged their own improvement in English language skills and recognized 
their increased accuracy in workbook activities. 

VI. DISTANCE LEARNING: SELECTED ISSUES 

The Crossroads Cafe ESOL Adult Learning Program was designed to teach English to LEP 
adults primarily as a distance learning model. The data suggest that teachers in these pro- 
grams developed specific pedagogical practices and procedures to ensure that students 
had unfettered access to program materials and were provided periodic supervision by 
program staff. Instructors went to great lengths to ensure that the "student teacher" rela- 
tionship not be jeopardized and that a sense of "rapport" not be lost. When possible, 
teachers spoke with students personally to assess their progress and offer assistance; 
other times, they conversed by telephone. Detailed and conscientious record keeping was 
vitally important in order to establish a sense of continuity. 

VI. PROGRAM RECOMMENDATIONS 

The team of evaluators listened carefully to the opinions and advice of program participants 
and have formulated recommendations educators who decide to develop future Cross- 
roads Cafe programs. These suggestions include advice in the following areas: expansion 
of program materials; methods for the recruitment, selection, and placement of learners; 
content and format for the training and orientation of instructors; and procedures and 
practices for program implementation. 

The study clearly demonstrated the feasibility of using Crossroads Cafe as the basis of 
ESOL instruction. Moreover, the feasibility of using Crossroads Cafe to promote distance 
learning as a means of reaching out to heretofore unreachable learners was also demon- 
strated. The data support the hypothesis that ESOL distance learning with Crossroads 
Cafe can be as effective as classroom instruction with the same materials at a significantly 
reduced cost. 

However, three major issues need to be resolved before distance or hybrid learning 
programs can be institutionalized: [1] Teachers need to be trained in distance learning 
methods; [2] Program logistics require special attention to facilitate transfer of videos and 
text materials between instructors and teachers; and [3] A state reimbursement formula for 
service providers must be established that incorporates suitable incentives so that admini- 
strators, school districts, BOCES, and CBO’s will choose to conduct distance learning 
programs. 
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CROSSROADS CAFE 

An ESOL Adult Learning Program 



All this work is very interesting. It helped me very much with my 
English. It is very good because at the same time I see the video, I 
also read about the same subject in the book and do exercises. In the 
book, you can see how words are written and in the video you can see 
how they are pronounced. That is one of the things I like the most. 
That is an excellent idea of Crossroads Cafe. Thank you. (A Cross- 
roads Cafe learner) 



I. INTRODUCTION 

This report presents the findings of a formative evaluation of the implementation of Cross- 
roads Cafe, An ESOL Program for Adult Learners, a distance learning adult level video 
program designed to teach English to Speakers of Other Languages (ESOL). The study 
design focuses on process: how the program was utilized by administrators, teachers, and 
learners rather than on how effective it was in improving English language skills. Program- 
matic and pedagogical procedures and practices were examined in an effort to understand 
"what worked, what didn't work, and why." The effectiveness of the Crossroads Cafe 
materials themselves were only peripherally addressed and were based on the perceptions 
of program participants. 

The Crossroads Cafe program is a series of 26 half-hour video episodes that tell the story 
of six ethnically diverse characters whose lives intersect at Crossroads Cafe, a neighbor- 
hood restaurant. Each episode contains two interrupting documentary-style segments that 
teach language and culture in a more direct manner. In addition, collateral print materials 
include Photo Stories, a multi-level Worktext, a Teacher's Resource Book, and a Partner 
Guide. 

The program is a the result of a three year collaborative initiated by a partnership of the 
private and public sectors to meet the English language needs of the millions of Limited 
English Proficient (LEP) adults in the United States. These parties included the United 
States Department of Education; the State Education Departments of California, Florida, 
Illinois, and New York; a partnership of INTELECOM (a California based not-for-profit 
production company); and Heinle and Heinle Publishers (language learning specialists). 

This formative evaluation of Crossroads Cafe was conducted by the Center for Advanced 
Study in Education (CASE) of the City University of New York Graduate School for the 
Office of Workforce Preparation and Continuing Education of the New York State Educa- 
tion Department (SED). The pilot programs were sponsored by the SED Office of Work- 
force Preparation and Continuing Education as one of several undertakings designed to 
introduce Crossroads Cafe program materials to ESOL policy makers and practitioners as 
well as to determine its relative success as a distance learning model for ESOL instruction. 
Information gleaned from this study will be used to formulate guidelines for administrators 
and instructors who will be involved in subsequent implementations of Crossroads Cafe. 
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Twenty-two sites, serving 755 learners, participated in this pilot study, which was sched- 
uled for six to eight week periods between June 1 and September 30, 1996. The sites, 
which were geographically dispersed throughout New York State, were selected on the 
basis of demonstrated success in prior ESOL instruction. 

Evaluation data, in full or in part, were obtained from 22 sites. Sixteen administrators from 
16 sites, 24 instructors from 15 sites, and 488 learners from 19 sites completed requests 
for information, a noteworthy contribution in view of the time constraints imposed by the six 
to eight week pilot period and the amount and complexity of the material requested. 

II. BACKGROUND AND CONTEXT OF THE CROSSROADS CAFE PROGRAM 
A. Historical Background 1 

The United States is a nation of immigrants, admitting as many legal immigrants and 
refugees annually as the rest of the world combined. The foreign born population, now at 
21 .2 million people, is larger than at any other time in our country's history. During the 
1980's, nearly 10 million legal and undocumented immigrants entered the United States, 
accounting for one-third of the population growth during that period. During the 1990's, 
between nine and twelve million immigrants — a record number — are expected to enter 
the country. 

Nearly half of the immigrant population has serious difficulties with the English language. 
The 1990 Census reports that 25.5 million adults (age 18 or over) in this country speak a 
language other than English at home and, of these, an estimated 12 to 14 million are 
unable to speak, understand, or write English without difficulty. Consequently, a substan- 
tial proportion of the immigrant population is unable to participate fully in American society 
because of limited proficiency in English. 

Moreover, this population is projected to increase at an average annual rate of 15 percent. 
By the turn of the century, an estimated 17.4 million Limited English Proficient (LEP) adults 
will reside in the United States. It is expected that 53 percent of the largest metropolitan 
areas in the country will have majority populations whose native language is not English. 

To meet the needs of the emerging LEP population, ESOL education has become the 
fastest growing sector of adult education in the United States. Between 1985 and 1995, 
enrollment in ESOL programs increased by more than 73 percent. However, existing 
programs are, and will continue to be, unable to meet the demand for ESOL education 
because of reductions in federal, state, and local budget allocations. Classrooms are 
overcrowded, budgets are stretched, and waiting lists are long. Although a substantial 
number of people — approximately 1.8 million — are served each year, this figure repre- 
sents fewer than 15 percent of the 14 million people who currently require assistance. 




1 Source: NYS Education Department, Office of Workforce Preparation and Continuing 
Education, Albany, New York, 1996 
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New York State is third in the nation with regard to the number of LEP adults who would 
benefit from ESOL instruction. Although enrollments in New York State ESOL programs 
have increased by more than 73 percent in the past decade, during that period ESOL 
services were provided to only 633,000 out of approximately 4.25 million in need. 

B. Description of the Program 

Crossroads Cafe is designed to teach English to adult learners working independently. 
This style of distance learning is conducted without an instructor. Customarily, an introduc- 
tory classroom session is required of learners to ensure proper use of the materials. After 
the initial session, contact with a professional teacher is usually by phone. Occasionally, 
short private sessions may be scheduled for assessment or assistance purposes. 

The target population includes adults who are literate in their native language and who 
have some proficiency in both spoken and written English as measured by the Basic 
English Test (BEST) levels 4, 5, and 6, or its equivalent. 2 

The Crossroads Cafe program design incorporates the following features: 

• A flexible multi-level approach. Crossroads Cafe print materials are de- 
signed to meet the needs of learners at low-beginning through high- 
intermediate levels of English. Learners work progressively by using the 
same video series and Worktext at each successive level. 

• Context-based learning. Crossroads Cafe story-lines portray situations 
that are relevant and meaningful to a foreign born population based on 
the premise that the importance of language to learners is determined by 
the situations in which language will be used. 

• Self-instructional methods. The collateral print materials were designed 
for independent learners who might not have access to more traditional 
ESOL classes. To ensure that they are user friendly, technical language 
is minimally used. Self-checks and an associated answer key are pro- 
vided for each unit. And students are encouraged to work with partners. 

• Communication skills. The materials enable users to develop communi- 
cative skills by focusing first on the purpose of communication and then 
on the forms of communication, both written and spoken. 

• Facilitation of different learning styles. Activities are designed to appeal 
to different kinds of learners. For example, those who prefer exercises 



2 The Basic English Skills Test (BEST) is a criterion-referenced test that yields scores that are 
linked to a taxonomy of specified reading and writing proficiencies. Scores can be readily translated to 
other English proficiency scales, such as the California Model Standards Proficiency Levels and other pro- 
ficiency levels of the New York State Placement Test. Center for Applied Linguistics, Washington, DC . 
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that elicit more objective "right-or-wrong" answers may complete the fill- 
in-the-blank or matching questions on sentence-sequencing or on read- 
ing comprehension. Others may prefer the activities that elicit self-ex- 
pression, the development of fluency, the development of meta-cog nitive 
skills, or reflection on the learning process. 

The Crossroads Cafe ESOL Adult Learning Program consists of five major components: 

• Video The main component is a series of 26 half-hour video episodes 
that provide education through entertainment. The videos," which are 
closed-captioned, portray the story of six ethnically diverse- characters 
whose lives intersect at Crossroads Cafe, a neighborhood restaurant. 

Each video episode contains two imbedded instructional segments that 
are directly related to the episode’s theme: 

■ Wordplay, which demonstrates a specific language function 
(e.g., expressing a person’s ability to do something) and the 
language structures necessary to express that function (e.g., 
can, know how to). 

■ Culture Clips, a documentary-style segment, which examines 
issue-oriented social or cultural themes that are dramatized in 
the story. 

• Worktext Twenty-six units contain exercises designed to develop story 
comprehension, language skills, and higher order thinking for intermedi- 
ate level learners. Three graduated levels of exercises are provided, 
including high beginning (focusing on words and phrases), low intermedi- 
ate (focusing on phrases and sentences), and high intermediate (focusing 
on discourse and paragraphs). 

• Photo Stories Simple dialogue balloons and four color photographs pre- 
sent a condensed pictorial tour of each episode for lower level learners. 

• Teacher Resource Package To maximize the benefits of the program, 
two resource books are available for teachers which include general 
instructions on program methods, episode-specific suggestions for 
optional enrichment activities, and fifty-two reproducible masters that 
provide interactive opportunities for students in a distance-learning 
program with a classroom component. 

• Partner Guide A guide with reproducible masters is available that can 
be used with a teacher or an English proficient friend or family member. 

The material provided during the New York State Crossroads Cafe pilot included the first 
eight video episodes and the collateral Photo Stories and Worktext. However, because 
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of summer scheduling time constraints, pilot programs conducted by the New York City 
Board of Education included the first six episodes only. 

III. DESCRIPTION OF THE IMPLEMENTATION EVALUATION STUDY 
A. Selection of study sites 



The Crossroads Cafe ESOL Adult Learning Program was piloted in 22 demonstration sites 
in six regions of the State, as shown in Table 1: 

TABLE 1 Mv 

Geographic Distribution of 22 Pilot Sites 



Western New York 

Erie County 1 

Niagara County 2 

Long Island 

Nassau County 1 

Suffolk County 2 

Mid-Hudson 

Orange-Ulster County 1 

Rockland County 1 

Westchester County 1 



New York City 

Bronx County . 1 

Kings County 2 

New York County 4 

Queens County 3 

Southern Tier 

Tomkins County 1 

Finger Lakes 

Monroe County 2 



Selection of sites was based on information provided by the New York State Education 
Department regarding the amount and quality of the sites’ prior ESOL experience as 
assessed by NYSED. Participation by the agency was voluntary. Location was also a 
consideration in selection to ensure statewide representation of sites. 



Teachers at each site were nominated by their administrators on the basis of the teachers’ 
experience and expertise in ESOL instruction; in selected sites, experience and expertise 
in distance-learning instruction were also criteria for inclusion. 

B. Implementation models 

Pilot sites selected one, and in a few cases two, of the three program models for imple- 
mentation of the Crossroads Cafe program. The models were defined by how the program 
materials were used, as follows: 



• Traditional Classroom Model, where the teacher showed the videos in 
class and used the instructional materials as classroom texts. Learners 
were not expected to use the videos and texts at home. 

• Hybrid Model, where the teacher used materials in the classroom and 
learners used them both in class with a teacher and on their own, without 
an instructor present. 



• Distance Learning Model, where Crossroads materials were not used in 
the classroom. Learners used both videos and text materials on their 
own, almost entirely without the presence of an instructor. 

C. Implementation measures 

The study design incorporated both qualitative and quantitative approaches to data 
collection and analysis. The implementation evaluation was based on information collec- 
tion methods that relied on self-reports by program participants, which included both 
periodic reports throughout the course of the demonstration and retrospective reports at 
the completion of the demonstration period. Data collection instruments included self- 
administered questionnaires and interviews. 

The self-administered questionnaires included the following: 

■ Learner Demographic Form, which profiles the sex, age, spoken lan- 
guage^), ethnicity, employment status, and level of education of each 
learner. 

■ Student Record Sheet, which tracks students' use of the videos and 
written materials. 

■ Episode Response Form, which describes students' utilization patterns 
and reactions for each episode of Crossroads Cafe. 

■ Mail Survey: Administrators (post-pilot), which describes the program 
model, student population, program staff, utilization and effectiveness of 
materials, and the implementation procedures at each site. 

■ Mail Survey: Teachers (post-pilot), which describes students' response 
to Crossroads Cafe, the teachers perceived effectiveness of the materi- 
als, and the program's strengths and limitations. 

The interviews included the following: 

■ Administrator Interview (telephone), which explored administrators' initial 
experiences during the implementation of the program with regard to the 
program model, the materials, the teachers, and the students. 

■ Focus Groups: Administrators/Teachers at regional sites (post-pilot), which 
explored the procedures used in the development, implementation, and 
operation of Crossroads Cafe as a distance learning model and elicited 
recommendations for program replication. 
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IV. Pilot Programs 

A. Description of Program Models 

Program models — classroom, hybrid, or distance learning — were selected by the partici- 
pating provider agencies for implementation of the Crossroads Cafe program on the basis 
of several criteria, including practical considerations such as the availability of funding and 
other resources as well as more ideological considerations such as a commitment to 
particular teaching modalities or philosophies. Program models were implemented without 
rigid prescriptors; therefore, the models varied widely within each category. 

Four classroom programs, seven hybrid programs, and eleven distance learning programs 
were conducted at 22 sites during the pilot period. [See Table 2 for distribution of pro- 
grams.] It should be noted that four of the 22 sites implemented more than one program 
model: one site had a traditional classroom and a hybrid model; two sites conducted both 
hybrid and a distance learning models, and another, a classroom and a distance learning 
model. 



Graphic 2 

Distribution of Program Models 



# of 

MODEL Programs Percent 

Distance Learning 11 50.0% 

Hybrid Mode! 7 31.8% 

Classroom 4 18.2% 



Total 22 100.0% 



18 % 




Distance learning was the most prevalent model, accounting for 50 percent of the pro- 
grams. Variations among demonstration sites in the day to day operation of this model 
were largely determined by the proximity of learners to the instructional site. One program 
was described by the administrator as follows: 

Once the [Crossroads Cafe] program was explained, the student was given 
the book and the first video. The student would use the material at home and 
mail the video back in the postage paid envelope that was provided; the next 
video would then be mailed out [and the same procedure followed]. There 
was a 24-hour phone number with an answering machine that they could call 
if they needed help, and the teacher would return the call as soon as possible. 

Telephone or personal contact with the teacher was made [at least] once a 
week, for about a half hour each time. Assessment was based on personal 
contact and written reports of contact. [Assessment criteria] included im- 
provement in written and spoken English and [the use] of English in daily 

life. 
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The administrator of another program described a similar model: 



Students were sent the pre-assessment by mail and when it was returned, they 
were sent the appropriate workbook by mail. Students then picked up videos 
once a week [at the school or a designated library]. Students called their 
tutor or were called by their tutor once a week. These calls and the correc- 
tion of work averaged an hour in length per week of the teacher 's time [per 

student.] 

A different distance learning model, utilizing a fiber-optic network, is described below: 

Distance learning as we designed it enabled us to provide instruction at three 
sites that are miles apart, across a two-way, fully interactive, voice and 
motion video fiber-optic network. Distance learning classrooms are identi- 
cally equipped. Students were assessed using Crossroads Cafe [materials] 
and the teacher 's prepared test. 

The hybrid model, used in 32% of the programs, combined features of both distance 
learning and classroom learning, as illustrated below 

The class viewed the videos. They divided into groups of three or four and 
worked on the Worktext activities [without the presence of the teacher] and 
freely discussed the episodes. The teacher explained to the class that the 
project wanted to implement distance learning, so they had to pretend they 
were on their own, [see the video again on their own, and work on their own. 

During] the last hour, the class together with the instructor reviewed the 
exercises, went over the pronunciation, and addressed the distance learning 

experience. 

Another administrator wrote: 

Students viewed tape 1 together in class, followed by a discussion of how to 
use the books and materials. Students then borrowed a different tape each 
week for home use. The students met one to one and a half hours each week 
with the teacher to discuss the tapes. 

The traditional classroom model, representing 18% of the pilot programs, was described 
by one administrator, as follows: 

Students watched [the videos] with the teacher. The teacher directed the 
Worktext activities. Students were not able to do the Worktext independently. 
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B. Recruitment and Selection of ESOL Learners 

Graphic 3: Student Population Distribution 

Students were recruited for this pilot study primarily 
from adult education classes taken during the 1995- 
96 school year, including classes in ESOL, Adult 
Basic Education (ABE), or General Educational De- 
velopment (GED), or from waiting lists for these 
classes. The large majority (79 percent) came from 
prior ESOL classes; the rest, from GED (14 percent) 
or ABE (7 percent). Table 3 illustrates the distribu- 
tion of the origins of the study population. 

Teachers acknowledged that “no ESOL program fits the needs of every learner.” There- 
fore, they indicated that it was important to distinguish prospective students who were apt 
to benefit most from a distance learning setting from those better suited to a traditional 
classroom setting. For the pilot programs, teachers relied to some extent on their own 
intuition in the student selection process. While a student’s proficiency was undoubtedly 
a primary concern, more subjective personal and situational factors were also considered. 

Students and, in some cases, their friends and family members were invited by teachers 
to participate in this program by verbal and/or written announcements. At some sites, fliers 
were prepared in both English and foreign languages. 

All sites provided orientation sessions for prospective students to explain and demonstrate 
the use of both the videos and the print materials. The meetings were scheduled to 
accommodate as many students as possible and were held during regular class time and 
during specially designated hours, both day and evening. Interested learners were then 
screened using standardized tests such as BEST, or an equivalent and placed at the 
appropriate ESOL level. At some sites, learners selected appropriate levels for them- 
selves. These choices were predicated upon the “level of comfort” each felt regarding the 
supporting text. In a few cases, initial recruitment was limited to learners whose specific 
English proficiency levels matched instructional requirements. 



7 % 





Graphic 4: 

Placement of Crossroads Cafe 
Learners by ESOL Level 



Beginning 



The study population mainly included learners 
who were at the "intermediate" level with regard 
to ESOL (61 percent)), as reported by admini- 
strators (n=15). Program staff generally agreed 
that this was the optimal level for Crossroads 
Cafe learners. 

Intermediate 




61 % 
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Enrollment varied, ranging from 8 learners to 59 learners per site, with a mean of 28. The 
number of students in each program model is shown below. [See Table 5.] 



Graphic 5: Enrollment by 

Program Model 



MODEL 


# Students 


% 


Distance Learning 


265 


43.0% 


Hybrid 


271 


44.0% 


Classroom 


80 


13.0% 


TOTAL 


616 


100.0% 




The level of involvement in this program was remarkable given the extent of familial and 
personal responsibilities faced by participants. They were often frustrated that the de- 
mands of work, childcare, and/or homemaking interfered with their ability to devote ade- 
quate time and effort to their studies, complicated even more by their summer schedules. 
Yet, they remained "motivated" and "enthusiastic" according to both self and teacher 
reports. They remained cognizant of the level of commitment required in order to succeed. 
"This will take a lot of time to do it right and get the most out of it," said one participant. 

Overall, the retention rate was relatively high, with 82 percent of participants remaining for 
the duration of the program according to administrators. Retention was highest in distance 
learning programs (87%) as compared with hybrid programs (79%) and classroom 
programs (75%). 

The high retention rate may be due, in part, to the fact that all students had previous exper- 
ience in adult education courses, either ESOL or ABE. They were known by their instruc- 
tors prior to the start of the pilot and were recruited specifically for this program. All partici- 
pants were aware of the “pilot” nature of the program and were likely to feel a greater than 
normal commitment to ensuring its “success.” The short duration of the pilot programs was 
a likely contributing factor in enabling programs to retain their students. 

Learners who left the program appeared to do so reluctantly. As quoted by a teacher, the 
ambivalence of one student was clear: "Even if this had been easier, I would not have 
been able to find the time or energy to do this [program]. But, I want to!” 



C. Profile of ESOL Learners 



The typical learner was female, 37 years old, Spanish-speaking, educated (at least a 
graduate of a foreign high school), a relative newcomer to the United States (median time, 
3 years), and unemployed. 



Sex [Graphic 6] 

69% of learners were female and 31% male. 




31% 



69 % 
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Age [Graphic 7] 



6% of learners were age 
7% were 60 or older; 



Age 


Number 


Percent 


16-20 


28 


5.9% 


21 -24 


44 


9.2% 


25-44 


279 


58.6% 


45-59 


92 


19.3% 


60 and over 


33 


6.9% 


TOTAL 


476 


100.0% 



16-20; 9% were 21-24; 59% were 25-44; 19% were 45-59; and 

7% 6% 




Prior Education [Graphic 8] 

29% of learners were not high school graduates; 31% were high school graduates; 11% 
had some college education; 18% were college graduates; and 11% had some post- 
graduate education. 



Level of Education 


Number 


Percent 


Less than High School 


137 


29.0% 


High School Graduate 


147 


31.1% 


Some College 


52 


11.0% 


College Graduate 


85 


18.0% 


Graduate/Professional 

School 


52 


11.0% 


TOTAL 


473 


100.0% 




Employment [Graphic 9] 



67 percent were unemployed; 33 percent were employed. Those who were employed 
worked an average of 33 hours per week. 



Employment Status 
Employed 
Unemployed 
Total 



Number % 

161 33%% 

327 67% 

488 100 % 
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Country of Origin [Graphic 10] 

29% were from Caribbean countries; 19% were from the ex-Soviet Union; 16% were from 
South American countries; the rest were from other countries. 



Origin of Students 


Number 


% 


Caribbean Nations 


139 


29% 


Ex-Soviet Union 


90 


19% 


South American Nations 


75 


16% 


Mexico 


37 


8% 


China 


29 


6% 


Poland 


12 


3% 


Viet Nam 


10 


2% 


Japan 


10 


2% 


African Nations 


11 


2% 


Other 


63 


13% 



TOTAL 476 100% 




Native Language [Graphic 11] 



The most common native languages among the learners were Spanish 45%, Russian 20%, 
Chinese 7%. 



Native Language 


Number 


% 


Spanish 


216 


45.5% 


Russian 


95 


20.0% 


Chinese 


35 


7.4% 


French 


23 


4.8% 


Haitian Creole 


22 


4.6% 


Polish 


12 


2.5% 


Vietnamese 


11 


2.3% 


Japanese 


10 


2.1% 


Urdu 


9 


1 .9% 


Korean 


7 


1.5% 


Ukranian 


6 


1 .3% 


Other 


29 


6.1% 


TOTAL 


475 


100 . 0 % 




Duration of Residency [Graphic 12] 



27 percent had resided in the United States fewer than 12 months; 26 percent, 1-3 years; 
19 percent 3-6 years; and 28 percent, 
more than 6 years. 



28% 



27% 



o 

ERIC 



Duration of Residency Number Per Cent 



Less than 1 Year 


120 


27.0% 


/v - V 

PwjrT: 


1 to 3 Years 


116 


26.1% 


I rttjLaifes 


3 to 6 Years 


84 


18.9% 




More than 6 Years 


124 


27.9% 


wn- 


Total 


444 


100% 


1 9% Ns- 
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D. The Implementation of the Crossroads Cafe Program 

Fourteen administrators operating 22 Crossroads Cafe programs (64%) rated the "ease 
of initiating" the pilot at their sites, using a 5-point scale where 1 indicated "difficult" and 5 
indicated "easy." The mean rating was 3.3, suggesting a relatively easy implementation 
process. Neither 1 nor 5 was indicated by any respondent. [See Table 13] 

Graphic 13 

Ease of Initiating the Pilot Program 
as Perceived by Administrators 




Based on the mean ratings, the ease of implementation was similar for all program models. 
Administrators in both classroom and distance learning settings gave a mean rating of 3.3, 
while the mean for those in hybrid settings was 3.0. 

The only problem associated with program implementation was a "late start-up" in about 
half the sites caused by insufficient lead time, delayed recruitment of students, and/or 
limited availability of the video cassettes and print program materials. The time constraints 
imposed by the pilot period required that instructional staff assume responsibility for 
duplicating videos for distribution to their students, a time-consuming and often costly 
solution. The staffs commitment to the program and to their students was demonstrated 
by the staffs willingness to assume this additional responsibility. 

Aside from the inconvenience posed by this initial problem, program staff viewed the 
implementation process favorably. They were enthusiastic about the design and quality 
of the Crossroads Cafe program and its contribution to the field of ESOL instruction. More- 
over, they took great pride in pioneering this innovative program. 

The relative ease of the implementation process may be attributed to at least four factors: 
[1] the caliber of the instructional staff, [2] the resourcefulness and commitment of the pilot 
program administrators, [3] the professional staff training sessions that prepared all 
participants, and [4] the quality of the Crossroads Cafe program materials. 

• The instructional staff included experienced professionals with demonstrated 
expertise in ESOL instruction. Table 14 shows that 76% of instructors were 
described by administrators as "very experienced" in this area; 1 0% as "moder- 
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ately" experienced; and only 14%, as "minimally experienced." About one-third 
of all teachers were experienced in distance learning. 

Graphic 14 

Level of Experience of ESOL Instructors 

14% 

Number % 

Very Experienced 22 75.9% 

Moderately Experienced 3 10.3% 

Minimally Experienced 4 13.8% 

TOTAL 29 100.0% 



• The pilot program administrators selected the staff members, designed and 
implemented student recruitment procedures on very short notice, re-allocated 
financial resources to support the costs of mounting the programs, and played 
a liaison role between their teachers and the Crossroads Cafe office throughout 
the pilot period. 

• Fifty-one instructors and program administrators (100%) from all pilot sites parti- 
cipated in Crossroads Cafe orientation and training sessions conducted by 
NYSED at four regional centers across the state. Sessions lasted four hours 
and covered the following areas: 

[1] Philosophy and practice of distance learning; 

[2] Crossroads Cafe video program and text materials; 

[3] Structure, organization, and administration of the pilots; and 

[4] State funding policies and practices. 

Thirty-nine participants (66%) completed an evaluation form regarding the training. 
The sessions were well-received and described by all respondents as providing 
a “good overview of the Crossroads Cafe program and distance learning.” 
Respondents also felt that the session “answered (their) questions satisfactorily." 
According to one participant: "We were not only given useful information about 
the program, but we also learned from the experience of others how to best run 
[Crossroads Cafe]." The distance learning training component was particularly 
useful according to one participant who said: "I was impressed at how much 
better prepared and how much more [the trainers] knew about distance learning 
than I did." 

In addition, some sites provided their own staff development activities, which 
varied in regularity and formality. Topics raged from the general, such as "pro- 
gram planning and development" to the more specific, such as "the use of video 
equipment." 
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• The Crossroads Cafe video series and the supporting textbooks had already been 
documented as high quality ESOL educational materials prior to the summer of 
1996. The entertainment and production excellence of the videos and texts virtu- 
ally ensured a significant increase in the learner’s “time on task” by providing 
another strong motivational component to the economic and social opportunities 
that mastering English already promised. 

V. PROGRAM MATERIALS 

A. Impression of Crossroads Cafe Program Materials 

The overall reaction to the Crossroads Cafe program was overwhelmingly positive, based 
on feedback from program participants. Unlike more traditional approaches, Crossroads 
Cafe promoted "a combination of reading, writing, and listening skills training." 

For teachers, this was an opportunity to be "creative," with regard to both instructional 
methods and materials. They were better able to engage students in ESOL instruction 
because language learning was a more enjoyable and enduring experience. One learner 
explained: 'This is a great program because it is interesting. And, if it is interesting, we are 
enthusiastic to study." 

Teachers rated the extent to which learners liked the Crossroads Cafe materials. The 
results were striking. On a scale of 1 to 5, where 1 indicated "dislike" and 5 indicated 
"liked," the average rating by teachers (n=23) was 4.5. Essentially, all teachers indicated 
that students reacted favorably to the materials. Fifty-seven percent said students "liked 
the materials;" 39 percent said students "liked the materials somewhat.” Mean ratings 
were similarly high in all program models: classroom 4.5, hybrid 4.3, distance 4.7. 

Graphic 15 

Reaction of Learners to Crossroads Cafe Materials 
as Perceived by Teachers 



4% 




Videos 

Clearly, the heart of the Crossroads Cafe program was the video series. The videos were 
described by students in their written comments for teachers as "interesting," "enjoyable," 
"engaging," "compelling," and "very positive." Why were the videos so popular? Students 
especially liked the fact that the stories portrayed "real-life" experiences and that they could 
"identify" with the characters and their struggles. “They find a piece of themselves in these 
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people,” said one instructor. The videos taught viewers how to act more effectively as 
individuals, as members of a group, and as members of a society, as highlighted by the 
following comments: 

"This [video teaches] that anybody can come to the United States and start 
a new career." (Episode entitled: “Opening Day”) 

"I talked to my children that it is not good when Henry lied to his parents." 
(Episode entitled: “Growing Pains”) 

"I learned that if you want to do something, you can do it. It does not matter 
if you are from another country, you have to reach your goal." (Episode 
entitled: “Worlds Apart”) 

"I learned. ..that you don't have to be ashamed of the work you do or the 
person you are. You do not have to be successful or professional so that 
people will like you or love you." (Episode entitled: “Who's the Boss?”) 

"I learned to find ways to keep our home secure from burglars by having a 
neighborhood watch." (Episode entitled: "Lost and Found”) 

"I learned that in any business, the changes should be gradual and not fast, 
because many problems will arise if you try to change things too fast." (Epi- 
sode entitled: “Time is Money”) 

"I have learned that it is not only hard for me but for everyone who come 
from a different country. We all have different struggles. (Episode entitled: 

“Fish Out of Water”) 

"I learned that children need a lot of attention from parents more than mater- 
ial things." (Episode entitled: “Family Matters”) 

The above comments serve to illustrate the psycho-social benefits derived from the 
Crossroads Cafe program, which are perhaps secondary to the intended goal of improving 
English language proficiency. Crossroads Cafe materials also enabled students to learn 
the meaning and spelling of words and expressions within the context of meaningful life 
situations. They also developed relevant skills such as using dictionaries, newspapers and 
other resources for improving their communication skills and for seeking employment. 

Moreover, the videos provided a catalyst for conversation because they gave students 
something "in common" to talk about. "Conversation is a priority for ESOL students," 
explained on teacher. "It is important to get them talking," said another. A colleague 
elaborated: “The combination of video and text gives the students a jump start in 
communicating their thoughts and ideas to others. It [becomes] easier for [them] to 
express their opinions.” 

The use of accents was the only feature of Crossroads Cafe that was a source of concern 
to many viewers. While some indicated that the accents gave authenticity to the stories, 
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others were more apt to find them "confusing," at least initially. One student remarked: 
"So many accents! I'll never meet this many people with this many different pronunciations 
in my whole life." While other comments were more subtle, they did serve to illustrate the 
fact that students were troubled by the "different pronunciations." Some simply had 
difficulty understanding the dialogue; a few others expressed a real concern that they 
would learn to speak English with yet "another accent." In addition, some ethnic groups 
were "irritated" by the portrayal of the characters as "uneducated" or of "lower socio- 
economic status" and had difficulty identifying with them. 

However, students soon adjusted to the accents. One teacher said that her students 
“learned to like the program very much because it gave them the opportunity to listen to 
different accents and ... to improve their comprehension of English as it is actually spoken." 

Generally, teachers did not share the view that accents were problematic. On the contrary, 
the suggested that learners could benefit from hearing different ways of speaking. One 
teacher explained: "Some people will always be uncomfortable with foreigners, but this 
program is good because it breaks [this attitude] down." Teachers felt that in their class- 
rooms, as well as in the community at large, people with different ethnic and cultural back- 
grounds are regularly brought together. Another, less frequently noted concern was that 
the characters spoke "too fast," but again learners adjusted to that in time as well. 

Texts 

The multi-level Worktext and Photo Stones were viewed by both learners and teachers as 
useful and welcome adjuncts to the video series. The Worktext, with its "tri-level" approach, 
enabled students to practice written English at a pace that was comfortable and/or chal- 
lenging for them. The multi-level work fostered self-confidence among the learners as they 
witnessed their own growth in English proficiency. Photo Stories, designed for use by high 
level one and level two students, was also effective as a "warm-up" prior to viewing the 
videos, or as means of review to "jog" one's memory after viewing the video for level three 
and four students. Students and teachers felt that these materials contributed to 
improvement in vocabulary, spelling, comprehension, and, to a lesser extent, grammar. 

Professional Resource Materials 

Not all Crossroads Cafe program materials were available to teachers during the pilot pro- 
gram. Several mentioned they would have benefited from the Teacher’s Resource Book, 
the Blackline Masters, and the Partner Guide. Nevertheless, they used their ingenuity and 
devised and/or adapted Crossroads and other ESOL materials to meet their particular 
instructional needs. For example, one distance learning teacher developed a procedure 
for screening students that involved mailing the NYS Place Test pictures to students and 
completing the remainder of the test over the phone. Some teachers used the Crossroads 
story line to develop exercises that would enhance language skills such as grammar, 
vocabulary, or writing. For instance, one exercise asked students to write alternative story 
endings. Teachers also developed ancillary materials geared to special student popula- 
tions. For example, one teacher working with Even Start families developed instructional 
materials related to parent literacy, employment, and culture. 

Finally, some instructors expanded the use of Crossroads Cafe materials themselves to 
meet the situational needs of their students. The development of employment skills was 
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a popular topic at some sites and one teacher described how she helped a students 
prepare for a job interview:' 

One student had to stop [Crossroads Cafe] because she was busy looking for a job. 

I gave her additional work, like [a copy] of an [ employment interview]. I went over 
the first tape with her to let her fully understand how to act on an interview. She 
finally got a job after several rejections. Later, she told me that the tape helped her 
a lot. 

A late addition to the pilot undertaking was the inclusion of an Adult Basic Education (ABE) 
class for native English speakers for the purpose assessing the appropriateness of Cross- 
roads materials with that population. The instructor reported that students found the mater- 
ials “eye catching,” but not academically challenging. ABE students are bent on improving 
their reading and writing skills in order to raise their GED scores. They are less concerned 
with improving conversational skills. Moreover, the materials dealing with acculturation and 
socialization were irrelevant to their needs. 



B. Utilization of Program Materials 
Where were the materials used? 

Crossroads Cafe program materials were designed to be utilized in a variety of ways — in 
a traditional classroom setting, outside the classroom, and/or in a combination of the two. 
Obviously, classroom use was not an option for those in distance learning modalities; 
however, materials were to used some extent during scheduled meetings between learners 
and teachers. In all program models, a number of factors including personal preference, 
time and scheduling issues, availability of books, videos, and VCR equipment, and other 
issues influenced where the materials were used. 



Graphic 16 shows that overall, most viewing (58%) occurred outside the classroom. How- 
ever, when the three models are compared, viewing patterns change. [See Graphic 17.] 
Nearly all students in classroom programs (89%) watched videos in class. Those in dis- 
tance learning settings (79%) watched videos at home. Students in hybrid programs were 
more likely to watch videos in class than elsewhere (59%), but a sizeable number watched 
at home (33%). Very few reported watching videos both at home and in class. 

Graphic 16: Where Were Videos Watched? 
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Graphic 17: Viewing Sites by Program Model 





Program Model 


Classroom 


% 


Hybrid 


% 


Distance 


% 


In Class 


42 


89.4% 


234 


59.2% 


137 


16.8% 


Outside 


1 


2.1% 


130 


32.9% 


645 


79.1% 


Both 


4 


8.5% 


31 


7.8% 


33 


4.0% 


Total 


47 


100.0% 


395 


100.0% 


815 


100.0% 



Graphic 18 shows that overall most reading (68 percent) work was completed outside the 
classroom. Again, when the three models are compared, work patterns change. [See 
Graphic 19.] Classroom students were more likely to use their workbooks in class (55%) 
than elsewhere. A substantial proportion of these students (35%), however, did their work 
both in class and elsewhere. Students in hybrid programs were almost as likely to do their 
work in class (48%) as elsewhere (39%). Distance learners clearly did most of their work 
outside of class, as would be expected (85%). 

Graphic 18 

Where was Reading Done? 
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Graphic 19: Textbook Work Sites by Program Model 
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How difficult was the material? 



Generally, the video and written materials were not perceived as difficult by the students. 
Based upon a 5-point scale, where 1 indicates "very difficult" and 5 indicates "not difficult 
at all," the mean scores for videos ranged from 3.1 to 3.4 and the mean scores for books 
ranged from 3.0 to 3.5 for the 8 individual units. Overall, the videos and textbooks received 
an average rating of 3.2 for all units combined, with two-thirds of the units described as 
either "not difficult" or of "average" difficulty. For the most part, students in the three 
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models rated the materials similarly, although those in the classroom models found them 
slightly easier to use, with an average rating of 3.4 for the videos and 3.5 for the books. 
Books were also rated less difficult in hybrid programs, but the videos were not. 

Graphic 20: Perceived Level of Difficulty 
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Student Responses re: Worktext 
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Mean Ratings by Learners of Perceived Level of Difficulty 

(1 = very difficult; 5 = very easy) 
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The large majority of students worked independently, with the help of family or friends. At 
least 70 percent of learners reported that they had no help on any of the units. They did, 
however, rely to some extent on the answer key in completing written assignments, At 
least half and as many as two-thirds of learners used it to check their work for "all" or 
"some" of the questions in each unit. 



How much work was completed? 

In spite of the time constraints imposed by the pilot period and the personal constraints 
faced by many learners as they juggled family, work, and other responsibilities, a consider- 
able amount of the assigned work was completed. 

Video assignments, however, were more likely to be completed than written assignments. 
When all units are considered, 86 percent of the videos were watched in their entirety, 
while 56 percent of the workbook answers were completed in their entirety. [See Table 20.] 
When students in the three program models are compared, those in distance learning 
settings were the most diligent, with 95% watching all videos completely and 62% answer- 
ing all textbook questions. 

It is likely that video watching accounted for a substantial portion of the time devoted to 
each unit. Reports by teachers, as well as reports from students themselves, indicated that 
videos were typically watched more than once, and in some cases as many as six or seven 
times each. “I watched the video many time to help me understand it,” said one student. 
It is not clear to what extent the difficulties with accents, the speed of the dialogue, or other 
factors contributed to the need for repeated viewings. 

Graphic 21 

How Much Work Was Completed by Learners? 
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The distance learning results can be attributed to the opportunity these students had to 
work at their own rates and at convenient times and places. The classroom setting is, by 
definition, constraining in regard to both time and place. Moreover, while classroom 
instruction may provide more opportunity for group discussion of the units, these 
discussions generally decrease the time available in class for students to complete their 
individual written work. „ 

Time On Task 

One indicator of the degree to which learners applied themselves to acquiring English 
language skills is relative ’’time on task.” Total minutes spent with each unit may be an 
indication of the motivational value of the materials. Increased total minutes may relate to 
the ease with which the materials can be used, as well as the learner’s perception of 
his/her own success in mastering the materials. Additional “time on task” may be con- 
sidered beneficial, as it is likely to result in enhanced acquisition of language proficiency. 



Median time per unit spent by pilot participants in all program models was an hour and a 
half. When program models are compared, some notable differences are apparent. [See 
Graphic 22.] The median “time on task” per unit spent by students in classroom and dis- 
tance learning settings was 120 minutes, while in hybrid programs the median “time on 
task” was 90 minutes. 

Graphic 22: Time on Task per Unit by Program Model 
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Thirty Minute Intervals 



Graphic 22 graphically illustrates the comparative “time on task” patterns found among the 
learners in the three model types. Among the classroom model learners, 88.6% are found 
in the first two clearly defined one hour intervals. With little or no work outside of the class- 
room, the graph shows the preponderance of classroom learners at those intervals. Des- 
pite a median completion time among classroom learners of 120 minutes, 48.6% com- 
pleted each unit in 60 minutes or less. Only the precise separation of discrete classroom 
meetings extended the median to the full 120 minutes. 

Hybrid learners’ completion periods are more smoothly distributed through the intermediate 
time periods. Yet 90.1% of the hybrid model learners are found to have completed work 
on the individual units by 120 minutes as well and demonstrate no real increased time per 
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unit when compared with their classroom counterparts. The hybrid model median time 
spent per unit was 60 minutes, less than the median shown for classroom learners. 
However, the distribution curve indicates a slower rate of completion and a significant 
number of hybrid model students completing units on their own at the 90 minute mark and 
more than 5% completing at 150 minutes. Yet overall, the difference in time on task 
between classroom and hybrid model learners is not significant. 

On the same scale, the distribution of distance learners who completed their units was 
much wider, indicating a significantly slower fall off of time spent and a much longer “time 
on task” per unit than among the learners in other model programs. The median length of 
time spent by distance learners on each unit was 120 minutes, but the extension by 
learners to 180 minutes and more is clearly shown in the data and represents 16.9% of the 
distance learners, a truly significant number. 

The additional “time on task “ spent by distance learners may be the result of the inherent 
freedom that distance learning provides. Distance learners not only can choose when to 
study and how long to study, but they can select an environment that is comfortable and 
well suited to study. If materials are attractive, effective, and entertaining, distance 
learners are likely to be as highly motivated by the materials as they are to become fluent 
in their new language. 

In contrast with the standard measurement of contact hours as an indicator of time spent 
acquiring English language proficiency, “time on task” is certainly broader, less precise, 
and more likely subject to the vagaries of student reporting. Yet, contact hours, because 
they reflect only actual classroom activity, are also unrealistic as an indicator of student 
effort. Contact hours cannot reflect the wide variation in student study patterns, student 
needs, and student motivation that can be more readily inferred from ’’time on task” data. 

Contact hours are defined as the number of students in class multiplied by the total number 
of classroom hours of instruction for that class. Traditionally, adult education activities 
have been supported by state funds transferred to provider agencies via a contact hour 
formula. In a traditional classroom setting, contact hours reflect accurately the number of 
instructor hours and the number of students served. However, if the classroom setting is 
altered as in distance learning or in the hybrid model, a non-traditional formula must be 
devised that can accurately reflect the provider agency’s true level of effort. 

The best indicators might be a careful comparison of both measurements, in which relative 
tendencies are likely to appear. Inferences drawn from an assessment of both data are 
more likely to reveal how much effort was committed to acquiring English proficiency than 
either one alone. Additionally, "time on task” is likely to give a greater insight into the 
independent reaction of the learners’ experience with the program materials than any other 
single source of data. 

The collected data indicate that, on average, the instructional hours spent with students 
varied considerably by model. The traditional classroom model was structured so that 
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each instructor spent approximately 6 hours per week before an average size class of 20 
students for a seven week period. Calculated in the traditional manner, the total for this 
instructional configuration equaled 840 contact hours. 

Similarly, in the hybrid model, instructors spent 2 hours per week with each class that 
averaged 39 students for a seven week period. For this configuration, the total equaled 
546 contact hours. 

For the distance learning model, the instructor spent approximately 45 minutes per week 
alone with each student plus one two hour session with all students during the orientation 
session. Therefore, during the first week, 24 students met for 2 hours with the instructor, 
which equals 48 contact hours. During the first week, the instructor began meeting 
students individually. Each week the teacher spent 15 hours working with 20 students, one 
at a time. Contact hours per week, therefore, equaled 1 5 per week or 105 contact hours 
for seven weeks plus the initial 48 hours of orientation. Therefore, the total for a seven 
week distance learning program for 24 learners was 153 contact hours. 

In comparison, the total average pilot contact hours were as follows: 

Classroom Model [20 students] 840 Contact Hours = 42 hours/student 

Hybrid Model [39 students] 546 Contact Hours = 14 hours/student 

Distance Learning Model [20 students] ... 1 53 Contact Hours = 6 hours/student 

These totals are representative of instructional time spent by teachers in direct contact with 

their students. Under the New York State formula which reimburses costs based upon 
contact hour, the classroom model is by far the most expensive for the state. However, if 
hybrid or distance learning models can be as effective in teaching English with Crossroads 
Cafe videos and texts, then state costs could be reduced significantly. 

Although reports reveal a very wide range, measurements of “time on task” indicate that 
learners tend to spend approximately 25% more "time on task" in distance learning models 
than in a hybrid model or when confined to classroom instruction alone. The data also 
reveal a significantly higher percentage of unit and video viewing completion for distance 
learners. Yet, evidence indicates that for distance learners the reduced “contact hours" 
had no harmful effect on language acquisition. [See Graphics 23 through 28 and accom- 
panying explanatory remarks.] 

VI. PROGRAM OUTCOMES 

This evaluation study was designed to elicit information that would provide guidance to 
future ESOL teachers and to program administrators in how to design and conduct hybrid 
and distance learning Crossroads Cafe ESOL programs. 

Certainly, this study was not designed to measure program outcome in terms of improve- 
ment in English literacy. The study team and program participants agreed that the duration 
of the pilot period was too brief to make such an assessment. Clearly, we have seen that 
both teachers and learners responded well to the program materials. Whether or not the 
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videos and books contributed to improvement in English languages skills and proficiency 
remains speculative, and at this time conclusions can only be based on subjective experi- 
ential and anecdotal evidence. 

Program staff reported that students improved their English language skills during the 
course of the pilot. Administrators (n=15) were asked to rate the effectiveness of both 
videos and workbooks using a 5-point scale, where 1 indicated "not effective" and 5 
indicated "very effective." Videos received an average rating of 3.7, with 15 percent of 
respondent saying they were "very effective" and another 54 percent saying they were 
"effective. Workbooks received a similar rating of 3.6, with 8 percent of respondents saying 
they were "very effective" and 54 percent saying they were "effective." 

Graphic 23: Effectiveness of Crossroads Cafe Materials 
As Perceived by Administrators 
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Graphic 24: Effectiveness of Crossroads Cafe Videos by Model 
As Perceived by Administrators 
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Graphic 25: Effectiveness of the Textbooks by Model 
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Eighty-four percent of the administrators of the hybrid programs rated the videos as “very 
effective” or “effective," while seventy-two percent of the distance learning administrators 
and only fifty percent of classroom administrators rated them so. 

The administrators were somewhat more uniform in their ratings of the text materials. 
These materials were rated as “very effective” or “effective” by 75% of classroom program 
administrators, by 71% of hybrid program administrators, and 67% of distance learning 
administrators. 

Teachers were also asked to rate the effectiveness of program materials, books and videos 
combined, in helping students improve their language skills. Clearly, teachers had more 
opportunity than administrators to assess the performance of program participants. Using 
the same 5-point scale, the average rating was 3.9 with 13% of teachers (n=24) saying 
they were "very effective" and 75% saying they were "effective." [See Graphics 26 and 28.] 

Graphic 26: Effectiveness of Crossroads Cafe Program Materials 

As Perceived by Teachers 
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Teachers in the three program models rated the materials similarly. Classroom 4.1; hybrid 
3.9; distance learning 3.8. [Scale: 1 = not effective; 5 = very effective] 

Teachers were further asked to indicate the extent of improvement in English language 
skills made by their students as a result of the Crossroads Cafe program materials. Nearly 
nine out of ten teachers reported at least some improvement among their students. 
Improvement was noted more in classroom programs, based upon the mean ratings of 
teachers. 

Classroom teachers rated the level of improvement as 3.5 as compared to 3.0 in distance 
learning programs and 2.7 for those in hybrid programs. [Scale: 1 = no improvement; 5 = 
very substantial improvement; see Graphics 27 and 28.] 

One possible explanation for these differences may be that the classroom setting offers 
more opportunity for student assessment, hence a greater likelihood of noticing improve- 
ment among learners. 



Graphic 27 : Student Improvement in English Language Skills Using Crossroads 

Cafe As Perceived by Teachers 
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Graphic 28 : Mean Ratings of Teachers by Pilot Model 

[Scale: 1 = no or none; 5 = most or very] 
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One distance learning teacher described her students' progress: 

In the beginning, I would speak to my students in English and they would 
answer me in Spanish. It required a lot of encouragement on my part, but 
gradually they showed more and more confidence in speaking English after 
exposure to the program. 

Indicators of improvement in English language skills were identified by teachers (n=24) as 
follows: 



Indicators of Improvement % Saying Yes 

Learners said their language skills improved 54 

Workbook answers became more accurate 50 

Spoken English improved noticeably 42 

Written English improved noticeably 25 

Learners used the answer keys less often 8 



What effect did the program have on the learners’ families? 

One of the secondary unanticipated benefits of the Crossroads Cafe program, particularly 
as a distance learning model, was the promotion of family literacy. The ESOL learners and 
their teachers reported many cases in which the materials were shared with family and to 
a lesser extent friends. One teacher said: "Whole families seemed to enjoy [the videos] 
together. Parents have [told me] that everyone is improving their English as a result of the 
videos." Young children especially took an active role, either learning along with a parent 
or acting as a helper, "a very positive experience for all involved.” 

However, some participants regretted that family members were often too involved with 
other responsibilities to offer consistent assistance, although reports of families watching 
Crossroads Cafe together were common. Additionally, teachers suggested that cultural 
variations contributed to the level and type of family involvement that was found. 

VII. DISTANCE LEARNING: SELECTED ISSUES 

The Crossroads Cafe ESOL Adult Learning Program was designed to teach English to LEP 
adults, primarily as a distance learning model. The evaluation study presented in this 
report examined the implementation of Crossroads Cafe in three modalities — traditional 
classroom, hybrid, and distance learning — in order to explore its relative utility and 
effectiveness for ESOL instruction. However, the richness of the data has enabled us to 
take a more in-depth look at distance learning, both as an instructional method and as an 
organizational model. The data suggest that specific programmatic and pedagogical 
practices and procedures are required for the effective implementation of distance learning, 
either alone or as part of a hybrid model. The practices must be designed to ensure that 
students have unfettered access to the program materials and are supervised periodically 
by program staff. 

Basically, teachers often felt hindered by the lack of time and proximity with their students 
and their inability to readily establish "rapport" or "close personal relationships" with them. 
They felt that telephone contact, which was the basis of most teacher-student interaction 
in these sites, was not sufficient and somewhat more precarious than face-to-face contact. 
"On the telephone, students can be lost in the snap of a finger," said one teacher. In some 
cases, students made themselves unavailable to talk on the phone because they were 
insecure about their vocabulary and pronunciation and, therefore, ill at ease trying to 
maintain a conversation. In a few other cases, students became truly unavailable when 
their phones were disconnected and/or mail was not acknowledged. 

As a result, program staff went to great lengths to ensure that the essence of the traditional 
"student-teacher relationship" not be jeopardized in a distance learning setting. The 
consensus was that relatively frequent meetings, once every week or ten days, would be 
optimal in order to review the student's work, bolster conversational skills, and maintain a 
sufficiently close rapport. However, at sites where students were geographically dispersed 
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or where work or other responsibilities impeded such regularity, teachers met with students 
less often or not at all. Face-to-face meetings with one or more students were scheduled 
at designated times and places including the student's or teacher's home, the public library, 
or school building. 

When direct contact was not feasible, telephone calls were made at appointed times either 
from teacher to student or the reverse. At least one teacher did not limit her students to 
scheduled calls; rather, she encouraged them to call "whenever there was a problem." 
Another teacher increased the length of telephone calls, as explained by her administrator: 

The ESOL teacher [on my staff! feh that she could not live within the original model 
of a half hour per student. She expanded the time to one hour and still felt a need for 
additional time to correct all the work submitted by the students. She found the calls 
to require a great amount of time because she made use of both the Spanish and 
English language. 

One respondent emphasized the importance of allowing adequate telephone time. In her 
experience, she had found "a clear connection between [the amount of] teacher student 
contact and student success.” 

In some cases, the degree and type of contact was influenced by the ESOL level of the 
student, with lower level students requiring more frequent personal contact. Those 
students with less confidence or less independence required more personal contact. 

Teachers felt that students working independently should have a semblance of structure 
in their work. Therefore, they "set goals" for their students by defining the nature and 
amount of work that was expected and sought to establish a sense of responsibility for 
compliance with these goals. One teacher explained that she made a "verbal contract" 
with her students to the effect that she would give her time and effort as long as they were 
willing to meet her expectations. 

Detailed and conscientious record-keeping was vitally important in distance learning set- 
tings in order to establish a sense of continuity. Teachers kept comprehensive files on 
each student, including personal and background information, utilization of program 
materials, and progress in English proficiency. One teacher explained how she obtained 
and recorded "anecdotal" information about each of her students — "anything that would 
help me understand them,” she said. For example, she inquired about the health of a 
student's child or the outcome of a spouse's job search. Such interaction not only 
strengthened the student-teacher relationship, but it encouraged the development of good 
conversational skills as well. 

The administrative tasks associated with the duplication, distribution, and collection of 
videocassettes were, perhaps, the most tedious component of distance learning. Pains- 
taking and varied procedures had to be established for the dissemination of materials. 
These procedures included designated pick-up and drop-off locations, such as the school 
building, library, mall, or other public places. Pre-paid mail delivery to and from the site 
and in-person home delivery/pick up by the teacher were also methods of cassette 



distribution. In the more rural distance learning sites, where access to towns or cities was 
limited, one solution was to duplicate multiple units on extended cassette tapes to reduce 
the number of exchanges required. 

Finally, a number of schools made "support services" available to Crossroads Cafe stu- 
dents. For example, in addition to the assistance of their ESOL teachers, learners could 
meet or talk with guidance counselors, resource teachers, or other adult education 
personnel, as needed. One site established a 24-hour "hotline" that was linked to an 
answering machine. 

VIII. PROGRAM RECOMMENDATIONS 

Clearly, based on information provided by program participants, the implementation of the 
Crossroads Cafe pilot may be described as a success. Administrators and teachers report- 
ed relatively little difficulty in initiating the program. In fact, they welcomed the opportunity 
to pilot new teaching materials, techniques, and modalities and were gratified at the 
progress made by their students. Their demonstrated resourcefulness in this venture has 
undoubtedly paved the way for the more widespread implementation of this initiative. 

Learners were similarly enthusiastic about the program, particularly their ability to integrate 
home and classroom study. For many, ESOL instruction would not have been possible 
without this option. These learners also reported relatively little difficulty in adapting to the 
pilot program. Despite the frequent conflicting demands of school, work, and family, most 
learners diligently completed the prescribed workload, and many felt they improved their 
English proficiency. 

The dedication and skill of the administrators and teachers, the motivation and perse- 
verance of the ESOL learners, and the high-quality and compelling nature of the program 
materials helped ensure the successful implementation of the pilots. However, this 
evaluation would do a disservice to ESOL educators and learners if it focused only on the 
program success. One primary goal in conducting this study was to ascertain how best to 
implement Crossroads Cafe as a distance learning model for ESOL instruction. In this 
regard, the team of evaluators listened carefully to the opinions and advice of program 
participants who, during the course of the summer, discovered first hand how to make the 
Crossroads Cafe ESOL Adult Learning Program viable both in terms of its teaching 
methods and program features. Recommendations derived from this experience are 
reported as follows: 

RECOMMENDATIONS 
Crossroads Cafe Program Materials 

■ Program materials that address grammar and usage skills should be expanded, 
especially those geared to higher level learners. Work sheet exercises in English 
usage and tied to the Crossroads context would be a valuable aid in instruction. 
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■ Crossroads Cafe episodes should be available on audio cassette, primarily for those 
who have limited access to television and a VCR. In our opinion, the video component 
is essential to satisfactory use of Crossroads Cafe and cannot be replaced by audio 
alone. However, an audio tape and script would provide ancillary learner support. 

■ The Worktexts and Photo Stories should be designed so that written exercises are "re- 
movable" (e.g., perforated tear-out or looseleaf format) and can be given to instructors 
for correction. 

■ Some program administrators suggested that text materials consist of individually 
laminated sheets that can be erased and used again as a textbook, rather than using 
Worktext and Photo Stories as consumable commodities. 

■ A verbatim script for each video episode should be made available to instructors for use 
in developing ancillary classroom or homework exercises and to learners as a means 
of reinforcing what they have seen on the screen. 

■ A dictionary or glossary of key words, idioms, and other phrases found in the program 
materials should be made available to students for reference. 

■ Multiple episodes should be made available on a single video cassette, as an 
alternative to the single episode version, to facilitate the distribution of when weekly 
exchanges are precluded by logistical or scheduling difficulties. 

■ Devise a formula or method for determining the optimal number of videos and books 
that are required for distribution at program sites based on factors such as program 
model, number of students, method and frequency of distribution, etc. 

Recruitment, Selection, and Placement of Crossroads Cafe Learners 

■ Organize booths or distribute fliers at community based sites such as public schools, 
health facilities, churches, food markets, and street fairs where those who need ESOL 
instruction may be reached directly. 

■ Encourage learners to select the text level — either Photo Stories or Worktext — with 
which they feel most comfortable when working with Crossroads Cafe. 

■ Run multi-lingual advertisements in the local media such as cable television, radio, and 
newspapers. 

■ Develop standardized materials for screening, placing, and monitoring Crossroads Cafe 
students. Establish criteria for identifying the potentially successful distance learner. 

■ Provide an orientation for prospective learners that describes the Crossroads Cafe 
program, provides "hands-on" experience with the videos and print materials, and 
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enables participants to speak directly with ESOL teachers and students who have 
previously participated in the program. 

Professional Staff Training for Crossroads Cafe Instructors 

■ The pilot programs demonstrated the need for professional staff training for distance 
learning with Crossroads Cafe. Instructors whose past experience was almost exclu- 
sively in the classroom had difficulty transferring their skills to a distance learning mode. 
Staff training must be sufficiently intensive to provide instructors the same degree of 
comfort in a distance learning mode as they were accustomed to in a standard 
classroom situation. 

■ Conduct an orientation and training program that incorporates the following features: 

□ Sessions led by administrators, instructors, and, when possible, learners who 
have participated in exemplary Crossroads Cafe programs. 

□ Workshops offered to new Crossroads Cafe instructors on techniques for 
teaching in a distance learning setting, such as methods of effective telephone 
communication, systems for tracking and monitoring student progress. 

□ Intensive, supervised use of Crossroads Cafe materials in a “distance learning 
internship” environment. 

■ Provide on-going opportunities for exchange of information among Crossroads Cafe 
professionals via web sites, seminars, newsletters, etc. 

■ Provide fully developed log forms for students and teachers to record as accurately as 
possible hours spent working on Crossroads Cafe materials. These records will be 
essential in documenting “time on task” as part of New York State's non-traditional 
funding formula for Crossroads Cafe hybrid and distance learning programs. 

Program Implementation 

■ If financially feasible, enable learners to work with both texts in various configurations. 
The Photo Stories may precede the viewing of the episode and be used as an intro- 
duction or may be used later as a means of review. Some pilots reported that learners 
who worked at first with Photo Stories rapidly graduated to success with the Worktexts. 

■ Explore the different means available for distribution of the video cassettes to distance 
learners who cannot attend your center on any regular basis. Churches, local malls, 
schools, union halls, libraries and other more convenient sites may all be willing to be 
a repository for Crossroads cassettes. As a last resort, exchanging cassettes by mail 
may be the best practice. Have a plan before class begins. 

■ Organize learners to work on Crossroads Cafe materials cooperatively in groups either 
on or off site. 
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■ As an adjunct to the Crossroads Cafe texts and tapes, auxiliary instructional materials 
should be provided for the learners to strengthen further their English usage skills. 
Exercises that emphasize and teach English grammar and glossaries of American 
idioms should be part of every ESOL teacher’s arsenal. 

■ Teach the learners to be their own instructors in ESOL. Whatever group time is avail- 
able, especially early in the semester, the learners must be taught how to teach English 
to themselves using Crossroads Cafe materials. 

■ Plan to spend at least one half hour individually by phone or in person each week with 
each learner in a distance learning or hybrid teaching mode, whenever feasible. 

■ Encourage learners to watch the videos at home with their families present, if possible. 

■ Design telephone work sheets for use by instructors in distance learning that structures 
and standardizes the telephone conversation with learners regarding Crossroads Cafe 
problems and progress. 

■ Maintain copious notes in a Learner Log that will track activities, problems, and prog- 
ress, including personal data and other information that will encourage the development 
of a close rapport between learner and teacher. 

■ Arrange for face to face contact at least once, but more if possible, in even the most 
extreme distance learning situation. As a last resort, exchange photos or sponsor at 
least one social event that learners will likely be able to attend during the semester. 

■ Remind all instructors that their lessons will be more effective if they take the time to 
view the videos themselves before they assign them to their students for viewing. 

■ Although the videos need not be viewed in numerical sequence, viewing them in that 
order is favored, if feasible. However, after viewing episode number one, the remaining 
episodes need not be viewed sequentially. 

■ Instructors should maintain precise logs of “time on task” as reported by learners and 
as supplemented by instructional time spent with the teacher present. 

■ Prior to presenting the first episode of Crossroads Cafe, instructors should discuss with 
all learners the use of accented English as spoken by the Crossroads Cafe characters. 
Issues of added difficulty and the danger of learning “incorrect English pronunciation” 
should be discussed in terms of the realities of American society and the benefits that 
derive from becoming familiar with American English as it is commonly spoken. 

IX Methodological Note 

Quantitative data analysis was limited in some cases by the relatively small number of 

study participants and the even smaller number who responded to some of the survey 
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instruments. Whenever possible, both qualitative and quantitative responses in the analy- 
sis and ensuing discussion of the findings were incorporated into the study. Nevertheless, 
caution must be used in interpreting results and generalizations must be made with care. 

The research design was based on the assumption that each pilot site would implement 
only one program model. As such, each site was assigned an identification number for 
purposes of data entry and data processing. Four sites, however, did in fact report having 
two models each. Consequently, the analyses reflect the aggregate responses in these 
four models, i.e. responses may be counted twice — once for each program model. For 
purposes of piloting the research design and survey instruments, as well as the program 
models, the overall findings and conclusions have not been altered significantly. 
Subsequent tracking, monitoring, and research involving Crossroads Cafe programs need 
to collect data by program, rather than by site. Additional analysis may be performed on 
sites with single models only. 

X Implications for Further Research 

Subsequent Crossroads Cafe programs will be implemented in a variety of distance 
learning modes. Therefore, future research needs to focus on student outcomes in terms 
of distance learning practices, student characteristics, and policy changes that will foster 
distance learning. Future research needs to identify and document those distance learning 
practices that are most likely to produce student gains for students with particular educa- 
tional, social, and cultural characteristics. Further research is also warranted with popula- 
tions completely new to the adult education experience, as the pilot sites served only 
adults with prior exposure. 

Encouraging provider agencies to adopt distance learning practices introduces a difficult 
financial component to the relationship that currently exists between the state and the 
providers. The funding mechanisms available to New York State need to be re-examined 
to identify those that will help the state attain its educational goals while keeping within its 
budget. Even if future research shows that student achievement in distance learning is 
comparable to student achievement in the classroom setting, provider agencies still may 
be unable to adopt these practices. Since current New York State regulations reward 
increased contact hours, provider agencies could suffer increased financial hardship by 
implementing a distance learning structure that will reduce contact hours. Therefore, in 
these circumstances, the New York State needs to reconfigure the reimbursement formula 
to encourage distance learning instruction and to extend hybrid models in the direction of 
distance learning. 

SUMMARY 

In summary, the research goals of this evaluation study were effectively met, despite some 
of the methodological limitations described above. The Crossroads Cafe pilot study 
demonstrated the relative ease of establishing ESOL instruction programs using these 
materials in all three program models. Moreover, within the limitations of this study, the 
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effectiveness of Crossroads Cafe as a vehicle to promote ESOL distance learning was also 
demonstrated, thus providing a viable means of helping heretofore unreachable learners. 

However, three major issues still need to be resolved before distance or hybrid learning 
programs can be institutionalized: [1] Teachers need to be trained in distance learning 
methods; [2] Program logistics require special attention to facilitate transfer of videos and 
text materials between instructors and teachers; and [3] A state reimbursement formula for 
service providers must be established that adequately and fairly funds school districts, 
BOCES, and CBO’s which choose to conduct distance learning programs. 

By means of this formative evaluation, the implementation process of the Crossroads Cafe 
program was examined and its relative success as an instructional model was explored. 
The information provided by teachers, administrators, and students who participated in this 
pioneering effort will help others institutionalize Crossroads Cafe program with some 
measure of efficiency. 
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